CS 473: Algorithms. Ruta Mehta. Spring University of Illinois, Urbana-Champaign. Ruta (UIUC) CS473 1 Spring / 32
|
|
|
- Camron Houston
- 5 years ago
- Views:
Transcription
1 CS 473: Algorithms Ruta Mehta University of Illinois, Urbana-Champaign Spring 2018 Ruta (UIUC) CS473 1 Spring / 32
2 CS 473: Algorithms, Spring 2018 Universal Hashing Lecture 10 Feb 15, 2018 Most slides are courtesy Prof. Chekuri Ruta (UIUC) CS473 2 Spring / 32
3 Part I Hash Tables Ruta (UIUC) CS473 3 Spring / 32
4 Dictionary Data Structure 1 U: universe of keys with total order: numbers, strings, etc. 2 Data structure to store a subset S U 3 Operations: 1 Search/look up: given x U is x S? 2 Insert: given x S add x to S. 3 Delete: given x S delete x from S 4 Static structure: S given in advance or changes very infrequently, main operations are lookups. 5 Dynamic structure: S changes rapidly so inserts and deletes as important as lookups. Can we do everything in O(1) time? Ruta (UIUC) CS473 4 Spring / 32
5 Hashing and Hash Tables Hash Table data structure: 1 A (hash) table/array T of size m (the table size). 2 A hash function h : U {0,..., m 1}. 3 Item x U hashes to slot h(x) in T. Ruta (UIUC) CS473 5 Spring / 32
6 Hashing and Hash Tables Hash Table data structure: 1 A (hash) table/array T of size m (the table size). 2 A hash function h : U {0,..., m 1}. 3 Item x U hashes to slot h(x) in T. Given S U. How do we store S and how do we do lookups? Ruta (UIUC) CS473 5 Spring / 32
7 Hashing and Hash Tables Hash Table data structure: 1 A (hash) table/array T of size m (the table size). 2 A hash function h : U {0,..., m 1}. 3 Item x U hashes to slot h(x) in T. Given S U. How do we store S and how do we do lookups? Ideal situation: 1 Each element x S hashes to a distinct slot in T. Store x in slot h(x) 2 Lookup: Given y U check if T [h(y)] = y. O(1) time! Ruta (UIUC) CS473 5 Spring / 32
8 Hashing and Hash Tables Collisions unavoidable if T < U. Several techniques to handle them. Ruta (UIUC) CS473 5 Spring / 32 Hash Table data structure: 1 A (hash) table/array T of size m (the table size). 2 A hash function h : U {0,..., m 1}. 3 Item x U hashes to slot h(x) in T. Given S U. How do we store S and how do we do lookups? Ideal situation: 1 Each element x S hashes to a distinct slot in T. Store x in slot h(x) 2 Lookup: Given y U check if T [h(y)] = y. O(1) time!
9 Handling Collisions: Chaining Collision: h(x) = h(y) for some x y. Chaining/Open hashing to handle collisions: 1 For each slot i store all items hashed to slot i in a linked list. T [i] points to the linked list 2 Lookup: to find if y U is in T, check the linked list at T [h(y)]. Time proportion to size of linked list. y f s Does hashing give O(1) time per operation for dictionaries? Ruta (UIUC) CS473 6 Spring / 32
10 Hash Functions Parameters: N = U (very large), m = T, n = S Goal: O(1)-time lookup, insertion, deletion. Single hash function If N m 2, then for any hash function h : U T there exists i < m such that at least N/m m elements of U get hashed to slot i. Ruta (UIUC) CS473 7 Spring / 32
11 Hash Functions Parameters: N = U (very large), m = T, n = S Goal: O(1)-time lookup, insertion, deletion. Single hash function If N m 2, then for any hash function h : U T there exists i < m such that at least N/m m elements of U get hashed to slot i. Any S containing all of these is a very very bad set for h! Ruta (UIUC) CS473 7 Spring / 32
12 Hash Functions Parameters: N = U (very large), m = T, n = S Goal: O(1)-time lookup, insertion, deletion. Single hash function If N m 2, then for any hash function h : U T there exists i < m such that at least N/m m elements of U get hashed to slot i. Any S containing all of these is a very very bad set for h! Such a bad set may lead to O(m) lookup time! Ruta (UIUC) CS473 7 Spring / 32
13 Hash Functions Parameters: N = U (very large), m = T, n = S Goal: O(1)-time lookup, insertion, deletion. Single hash function If N m 2, then for any hash function h : U T there exists i < m such that at least N/m m elements of U get hashed to slot i. Any S containing all of these is a very very bad set for h! Such a bad set may lead to O(m) lookup time! Lesson: Consider a family H of hash functions with good properties and choose h uniformly at random. Guarantees: small # collisions in expectation for a given S. H should allow efficient sampling. Ruta (UIUC) CS473 7 Spring / 32
14 Universal Hashing Question: What are good properties of H in distributing data? Ruta (UIUC) CS473 8 Spring / 32
15 Universal Hashing Question: What are good properties of H in distributing data? 1 Uniform: Consider any element x U. Then if h H is picked randomly then x should go into a random slot in T. In other words Pr[h(x) = i] = 1/m for every 0 i < m. Ruta (UIUC) CS473 8 Spring / 32
16 Universal Hashing Question: What are good properties of H in distributing data? 1 Uniform: Consider any element x U. Then if h H is picked randomly then x should go into a random slot in T. In other words Pr[h(x) = i] = 1/m for every 0 i < m. 2 Universal: Consider any two distinct elements x, y U. Then if h H is picked randomly then the probability of a collision between x and y should be at most 1/m. In other words Pr[h(x) = h(y)] = 1/m (cannot be smaller). Ruta (UIUC) CS473 8 Spring / 32
17 Universal Hashing Question: What are good properties of H in distributing data? 1 Uniform: Consider any element x U. Then if h H is picked randomly then x should go into a random slot in T. In other words Pr[h(x) = i] = 1/m for every 0 i < m. 2 Universal: Consider any two distinct elements x, y U. Then if h H is picked randomly then the probability of a collision between x and y should be at most 1/m. In other words Pr[h(x) = h(y)] = 1/m (cannot be smaller). 3 Second property is stronger than the first and the crucial issue. Definition A family of hash function H is (2-)universal if for all distinct x, y U, Pr h H [h(x) = h(y)] = 1/m where m is the table size. Ruta (UIUC) CS473 8 Spring / 32
18 Analyzing Universal Hashing Question: What is the expected time to look up x in T using h assuming chaining used to resolve collisions? Answer: O(n/m). Ruta (UIUC) CS473 9 Spring / 32
19 Analyzing Universal Hashing Question: What is the expected time to look up x in T using h assuming chaining used to resolve collisions? Answer: O(n/m). Comments: 1 O(1) expected time also holds for insertion. 2 Analysis assumes static set S but holds as long as S is a set formed with at most O(m) insertions and deletions. 3 Worst-case: look up time can be large! How large? Ω(log n/ log log n) Ruta (UIUC) CS473 9 Spring / 32
20 Compact Universal Hash Family Parameters: N = U, m = T, n = S 1 Choose a prime number p > N. Define function h a,b (x) = ((ax + b) mod p) mod m. 2 Let H = {h a,b a, b Z p, a 0} (Z p = {0, 1,..., p 1}). Ruta (UIUC) CS Spring / 32
21 Compact Universal Hash Family Parameters: N = U, m = T, n = S 1 Choose a prime number p > N. Define function h a,b (x) = ((ax + b) mod p) mod m. 2 Let H = {h a,b a, b Z p, a 0} (Z p = {0, 1,..., p 1}). Note that H = p(p 1). Ruta (UIUC) CS Spring / 32
22 Compact Universal Hash Family Parameters: N = U, m = T, n = S 1 Choose a prime number p > N. Define function h a,b (x) = ((ax + b) mod p) mod m. 2 Let H = {h a,b a, b Z p, a 0} (Z p = {0, 1,..., p 1}). Note that H = p(p 1). Theorem H is a universal hash family. Ruta (UIUC) CS Spring / 32
23 Compact Universal Hash Family Parameters: N = U, m = T, n = S 1 Choose a prime number p > N. Define function h a,b (x) = ((ax + b) mod p) mod m. 2 Let H = {h a,b a, b Z p, a 0} (Z p = {0, 1,..., p 1}). Note that H = p(p 1). Theorem H is a universal hash family. Comments: 1 h a,b can be evaluated in O(1) time. 2 Easy to store, i.e., just store a, b. Easy to sample. Ruta (UIUC) CS Spring / 32
24 Some math required... Lemma (LemmaUnique) Let p be a prime number, and Z p = {0, 1,..., p 1}. x: an integer number in Z p, x 0 = There exists a unique y Z p s.t. xy = 1 mod p. In other words: For every element there is a unique inverse. = set Z p = {0, 1,..., p 1} when working modulo p is a field. Ruta (UIUC) CS Spring / 32
25 Proof of LemmaUnique Claim Let p be a prime number. For any x, y, z {1,..., p 1} s.t. y z, we have that xy mod p xz mod p. Ruta (UIUC) CS Spring / 32
26 Proof of LemmaUnique Claim Let p be a prime number. For any x, y, z {1,..., p 1} s.t. y z, we have that xy mod p xz mod p. Proof. Assume for the sake of contradiction xy mod p = xz mod p. Then x(y z) = 0 mod p = p divides x(y z) = p divides x OR p divides (y x) (why?) = y z = 0 = y = z And that is a contradiction. Ruta (UIUC) CS Spring / 32
27 Proof of LemmaUnique Lemma (LemmaUnique) Let p be a prime number, x: an integer number in {1,..., p 1}. = There exists a unique y s.t. xy = 1 mod p. Proof. By the above claim if xy = 1 mod p and xz = 1 mod p then y = z. Hence uniqueness follows. Ruta (UIUC) CS Spring / 32
28 Proof of LemmaUnique Lemma (LemmaUnique) Let p be a prime number, x: an integer number in {1,..., p 1}. = There exists a unique y s.t. xy = 1 mod p. Proof. By the above claim if xy = 1 mod p and xz = 1 mod p then y = z. Hence uniqueness follows. Existence. For any x {1,..., p 1} we have that {x 1 mod p, x 2 mod p,..., x (p 1) mod p} = Ruta (UIUC) CS Spring / 32
29 Proof of LemmaUnique Lemma (LemmaUnique) Let p be a prime number, x: an integer number in {1,..., p 1}. = There exists a unique y s.t. xy = 1 mod p. Proof. By the above claim if xy = 1 mod p and xz = 1 mod p then y = z. Hence uniqueness follows. Existence. For any x {1,..., p 1} we have that {x 1 mod p, x 2 mod p,..., x (p 1) mod p} = {1, 2,..., p 1}. = There exists a number y {1,..., p 1} such that xy = 1 mod p. Ruta (UIUC) CS Spring / 32
30 Proof of the Theorem: Outline h a,b (x) = ((ax + b) mod p) mod m). Theorem H = {h a,b a, b Z p, a 0} is universal. Proof. Fix x, y U. Show that Pr ha,b H[h a,b (x) = h a,b (y)] 1/m. Note that H = p(p 1). Ruta (UIUC) CS Spring / 32
31 Proof of the Theorem: Outline h a,b (x) = ((ax + b) mod p) mod m). Theorem H = {h a,b a, b Z p, a 0} is universal. Proof. Fix x, y U. Show that Pr ha,b H[h a,b (x) = h a,b (y)] 1/m. Note that H = p(p 1). 1 Let (a, b) (equivalently h a,b ) be bad for x, y if h a,b (x) = h a,b (y). Ruta (UIUC) CS Spring / 32
32 Proof of the Theorem: Outline h a,b (x) = ((ax + b) mod p) mod m). Theorem H = {h a,b a, b Z p, a 0} is universal. Proof. Fix x, y U. Show that Pr ha,b H[h a,b (x) = h a,b (y)] 1/m. Note that H = p(p 1). 1 Let (a, b) (equivalently h a,b ) be bad for x, y if h a,b (x) = h a,b (y). At most howmany bad h is ok? Ruta (UIUC) CS Spring / 32
33 Proof of the Theorem: Outline h a,b (x) = ((ax + b) mod p) mod m). Theorem H = {h a,b a, b Z p, a 0} is universal. Proof. Fix x, y U. Show that Pr ha,b H[h a,b (x) = h a,b (y)] 1/m. Note that H = p(p 1). 1 Let (a, b) (equivalently h a,b ) be bad for x, y if h a,b (x) = h a,b (y). At most howmany bad h is ok? 2 Claim: Number of bad (a, b) is at most p(p 1)/m. Ruta (UIUC) CS Spring / 32
34 Proof of the Theorem: Outline h a,b (x) = ((ax + b) mod p) mod m). Theorem H = {h a,b a, b Z p, a 0} is universal. Proof. Fix x, y U. Show that Pr ha,b H[h a,b (x) = h a,b (y)] 1/m. Note that H = p(p 1). 1 Let (a, b) (equivalently h a,b ) be bad for x, y if h a,b (x) = h a,b (y). At most howmany bad h is ok? 2 Claim: Number of bad (a, b) is at most p(p 1)/m. 3 Total number of hash functions is p(p 1) and hence probability of a collision is 1/m. Ruta (UIUC) CS Spring / 32
35 Intuition for the Claim g a,b (x) = (ax + b) mod p, h a,b (x) = (g a,b (x)) mod m First map x y to r = g a,b (x) and s = g a,b (y). LemmaUnique proof = r s x (x, y) y Ruta (UIUC) CS Spring / 32
36 Intuition for the Claim g a,b (x) = (ax + b) mod p, h a,b (x) = (g a,b (x)) mod m First map x y to r = g a,b (x) and s = g a,b (y). LemmaUnique proof = r s x r (x, y) y s (r, s) As (a, b) varies, (r, s) takes all possible p(p 1) values. Since (a, b) is picked u.a.r., every value of (r, s) has equal probability. Ruta (UIUC) CS Spring / 32
37 Intuition for the Claim g a,b (x) = (ax + b) mod p, h a,b (x) = (g a,b (x)) mod m r s (r, s) = mod m Ruta (UIUC) CS Spring / 32
38 Intuition for the Claim g a,b (x) = (ax + b) mod p, h a,b (x) = (g a,b (x)) mod m = mod m Ruta (UIUC) CS Spring / 32
39 Intuition for the Claim For a fixed a {0,..., m 1} what is an upper bound on the size of set {s {0,..., (p 1)} a = s mod m}? (A) m. (B) m 2. (C) p. (D) p/m. (E) Many. At least two. Ruta (UIUC) CS Spring / 32
40 Intuition for the Claim g a,b (x) = (ax + b) mod p, h a,b (x) = (g a,b (x)) mod m 1 First part of mapping maps (x, y) to a random location (g a,b (x), g a,b (y)) in the matrix. 2 (g a,b (x), g a,b (y)) is not on main diagonal. 3 All blue locations are bad map by mod m to a location of collision. 4 But... at most 1/m fraction of allowable locations in the matrix are bad. Ruta (UIUC) CS Spring / 32
41 We need to show at most 1/m fraction of bad h a,b 2 lemmas... h a,b (x) = (((ax + b) mod p) modm) Fix x y Z p, and let r = (ax + b) mod p and s = (ay + b) mod p. Ruta (UIUC) CS Spring / 32
42 We need to show at most 1/m fraction of bad h a,b 2 lemmas... h a,b (x) = (((ax + b) mod p) modm) Fix x y Z p, and let r = (ax + b) mod p and s = (ay + b) mod p. 1 1-to-1 correspondence between p(p 1) pairs of (a, b) (equivalently h a,b ) and p(p 1) pairs of (r, s). Ruta (UIUC) CS Spring / 32
43 We need to show at most 1/m fraction of bad h a,b 2 lemmas... h a,b (x) = (((ax + b) mod p) modm) Fix x y Z p, and let r = (ax + b) mod p and s = (ay + b) mod p. 1 1-to-1 correspondence between p(p 1) pairs of (a, b) (equivalently h a,b ) and p(p 1) pairs of (r, s). 2 Out of all possible p(p 1) pairs of (r, s), at most p(p 1)/m fraction satisfies r mod m = s mod m. Ruta (UIUC) CS Spring / 32
44 Some Lemmas Lemma If x y then for any a, b Z p such that a 0, we have ax + b mod p ay + b mod p. Ruta (UIUC) CS Spring / 32
45 Some Lemmas Lemma If x y then for any a, b Z p such that a 0, we have ax + b mod p ay + b mod p. Proof. Suppose not ax + b mod p = ay + b mod p a(x y) mod p = 0 Ruta (UIUC) CS Spring / 32
46 Some Lemmas Lemma If x y then for any a, b Z p such that a 0, we have ax + b mod p ay + b mod p. Proof. Suppose not ax + b mod p = ay + b mod p a(x y) mod p = 0 Since p is a prime, p divides either a or (x y). Ruta (UIUC) CS Spring / 32
47 Some Lemmas Lemma If x y then for any a, b Z p such that a 0, we have ax + b mod p ay + b mod p. Proof. Suppose not ax + b mod p = ay + b mod p a(x y) mod p = 0 Since p is a prime, p divides either a or (x y). But a < p and (x y) < p, and hence a = 0 or (x y) = 0. Contradiction! Ruta (UIUC) CS Spring / 32
48 Some Lemmas Lemma If x y then for each (r, s) such that r s and 0 r, s p 1 there is exactly one a, b such that ax + b mod p = r and ay + b mod p = s. Proof. Solve the two equations: ax + b = r mod p and ay + b = s mod p Ruta (UIUC) CS Spring / 32
49 Some Lemmas Lemma If x y then for each (r, s) such that r s and 0 r, s p 1 there is exactly one a, b such that ax + b mod p = r and ay + b mod p = s. Proof. Solve the two equations: ax + b = r mod p and ay + b = s mod p We get a = r s x y mod p and b = r ax mod p. One-to-one correspondence between (a, b) and (r, s) Ruta (UIUC) CS Spring / 32
50 Understanding the hashing Once we fix a and b, and we are given a value x, we compute the hash value of x in two stages: 1 Compute: r (ax + b) mod p. 2 Fold: r r mod m Ruta (UIUC) CS Spring / 32
51 Understanding the hashing Once we fix a and b, and we are given a value x, we compute the hash value of x in two stages: 1 Compute: r (ax + b) mod p. 2 Fold: r r mod m Collision... Given two distinct values x and y they might collide only because of folding. Lemma # not equal pairs (r, s) of Z p Z p that are folded to the same number is p(p 1)/m. Ruta (UIUC) CS Spring / 32
52 Folding numbers Lemma # pairs (r, s) Z p Z p such that r s and r mod m = s mod m (folded to the same number) is p(p 1)/m. Proof. Consider a pair (r, s) {0, 1,..., p 1} 2 s.t. r s. Fix r: 1 a = r mod m. Ruta (UIUC) CS Spring / 32
53 Folding numbers Lemma # pairs (r, s) Z p Z p such that r s and r mod m = s mod m (folded to the same number) is p(p 1)/m. Proof. Consider a pair (r, s) {0, 1,..., p 1} 2 s.t. r s. Fix r: 1 a = r mod m. 2 There are p/m values of s that fold into a. That is r mod m = s mod m. 3 One of them is when r = s. 4 = # of colliding pairs Ruta (UIUC) CS Spring / 32
54 Folding numbers Lemma # pairs (r, s) Z p Z p such that r s and r mod m = s mod m (folded to the same number) is p(p 1)/m. Proof. Consider a pair (r, s) {0, 1,..., p 1} 2 s.t. r s. Fix r: 1 a = r mod m. 2 There are p/m values of s that fold into a. That is r mod m = s mod m. 3 One of them is when r = s. 4 = # of colliding pairs ( p/m 1)p (p 1)p/m Ruta (UIUC) CS Spring / 32
55 Proof of Claim # of bad pairs is p(p 1)/m Proof. Let a, b Z p such that a 0 and h a,b (x) = h a,b (y). 1 Let r = ax + b mod p and s = ay + b mod p. 2 Collision if and only if r mod m = s mod m. 3 (Folding error): Number of pairs (r, s) such that r s and 0 r, s p 1 and r mod m = s mod m is p(p 1)/m. 4 From previous lemma there is one-to-one correspondence between (a, b) and (r, s). Hence total number of bad (a, b) pairs is p(p 1)/m. Ruta (UIUC) CS Spring / 32
56 Proof of Claim # of bad pairs is p(p 1)/m Proof. Let a, b Z p such that a 0 and h a,b (x) = h a,b (y). 1 Let r = ax + b mod p and s = ay + b mod p. 2 Collision if and only if r mod m = s mod m. 3 (Folding error): Number of pairs (r, s) such that r s and 0 r, s p 1 and r mod m = s mod m is p(p 1)/m. 4 From previous lemma there is one-to-one correspondence between (a, b) and (r, s). Hence total number of bad (a, b) pairs is p(p 1)/m. Prob of x and y to collide: # bad (a, b) pairs #(a, b) pairs = p(p 1)/m p(p 1) = 1 m. Ruta (UIUC) CS Spring / 32
57 Look up Time Say S = T = m. For 0 i m 1, l(i) : list of elements hashed to slot i in T. Expected look up time Since for x y, Pr [ h a,b (x) = h a,b(y) ] = 1/m, we get E[ l(i) ] = S /m 1. Ruta (UIUC) CS Spring / 32
58 Look up Time Say S = T = m. For 0 i m 1, l(i) : list of elements hashed to slot i in T. Expected look up time Since for x y, Pr [ h a,b (x) = h a,b(y) ] = 1/m, we get E[ l(i) ] = S /m 1. Expected worst case look up time [ Like in Balls & Bins, E max m 1 i =0 l(i) ] O(ln n/ ln ln n). Ruta (UIUC) CS Spring / 32
59 Look up Time Say S = T = m. For 0 i m 1, l(i) : list of elements hashed to slot i in T. Expected look up time Since for x y, Pr [ h a,b (x) = h a,b(y) ] = 1/m, we get E[ l(i) ] = S /m 1. Expected worst case look up time [ Like in Balls & Bins, E max m 1 What if T = m 2 (# Bins is m 2 ) [ Claim: If T = m 2, then E i =0 l(i) ] O(ln n/ ln ln n). max m 1 i =0 l(i) ] = O(1). Ruta (UIUC) CS Spring / 32
60 Perfect Hashing Two levels of hash tables Question: Can we make look up time O(1) in worst case? Perfect Hashing for Static Data Do hashing once. If Y i = l(i) > 10 then hash elements of l(i) to a table of size Y 2 i. Ruta (UIUC) CS Spring / 32
61 Perfect Hashing Two levels of hash tables Question: Can we make look up time O(1) in worst case? Perfect Hashing for Static Data Do hashing once. If Y i = l(i) > 10 then hash elements of l(i) to a table of size Y 2 i. Lemma (Look-up) Expected worst case look up time is O(1). Ruta (UIUC) CS Spring / 32
62 Perfect Hashing Two levels of hash tables Question: Can we make look up time O(1) in worst case? Perfect Hashing for Static Data Do hashing once. If Y i = l(i) > 10 then hash elements of l(i) to a table of size Y 2 i. Lemma (Look-up) Expected worst case look up time is O(1). Lemma (Size) If S = O(m) then space usage of perfect hashing is O(m). Ruta (UIUC) CS Spring / 32
63 Intuition: Throwing m Balls in to m 2 Bins Pr[ith ball lands in jth bin] Ruta (UIUC) CS Spring / 32
64 Intuition: Throwing m Balls in to m 2 Bins Pr[ith ball lands in jth bin] = 1/m 2 For a fixed bin j, Y j =# balls in bin j. Ruta (UIUC) CS Spring / 32
65 Intuition: Throwing m Balls in to m 2 Bins Pr[ith ball lands in jth bin] = 1/m 2 For a fixed bin j, Y j =# balls in bin j. E[Y j ] = 1/m. Ruta (UIUC) CS Spring / 32
66 Intuition: Throwing m Balls in to m 2 Bins Pr[ith ball lands in jth bin] = 1/m 2 For a fixed bin j, Y j =# balls in bin j. E[Y j ] = 1/m. For c 3, let δ = cm 1. Pr[Y j > c]? Ruta (UIUC) CS Spring / 32
67 Intuition: Throwing m Balls in to m 2 Bins Pr[ith ball lands in jth bin] = 1/m 2 For a fixed bin j, Y j =# balls in bin j. E[Y j ] = 1/m. For c 3, let δ = cm 1. Pr[Y j > c]? Pr[Y j > cm/m] = Pr[Y ( j > (1 ) + δ) E[Y j ]] µ e (Chernoff ) < δ (1+δ) ( (1+δ) ) 1/m e = (cm 1) (cm) (e/c) c (1/m c ) cm 1/m 3 Ruta (UIUC) CS Spring / 32
68 Intuition: Throwing m Balls in to m 2 Bins Pr[ith ball lands in jth bin] = 1/m 2 For a fixed bin j, Y j =# balls in bin j. E[Y j ] = 1/m. For c 3, let δ = cm 1. Pr[Y j > c]? Pr[Y j > cm/m] = Pr[Y ( j > (1 ) + δ) E[Y j ]] µ e (Chernoff ) < δ (1+δ) ( (1+δ) ) 1/m e = (cm 1) (cm) (e/c) c (1/m c ) cm 1/m 3 [ ] Pr max m2 j=1 Y j > c Ruta (UIUC) CS Spring / 32
69 Intuition: Throwing m Balls in to m 2 Bins Pr[ith ball lands in jth bin] = 1/m 2 For a fixed bin j, Y j =# balls in bin j. E[Y j ] = 1/m. For c 3, let δ = cm 1. Pr[Y j > c]? Pr[Y j > cm/m] = Pr[Y ( j > (1 ) + δ) E[Y j ]] µ e (Chernoff ) < δ (1+δ) ( (1+δ) ) 1/m e = (cm 1) (cm) (e/c) c (1/m c ) cm 1/m 3 [ ] Pr max m2 j=1 Y j > c 1/m (Union bound). [ ] Pr max m2 j=1 Y j c Ruta (UIUC) CS Spring / 32
70 Intuition: Throwing m Balls in to m 2 Bins Pr[ith ball lands in jth bin] = 1/m 2 For a fixed bin j, Y j =# balls in bin j. E[Y j ] = 1/m. For c 3, let δ = cm 1. Pr[Y j > c]? Pr[Y j > cm/m] = Pr[Y ( j > (1 ) + δ) E[Y j ]] µ e (Chernoff ) < δ (1+δ) ( (1+δ) ) 1/m e = (cm 1) (cm) (e/c) c (1/m c ) cm 1/m 3 [ ] Pr max m2 j=1 Y j > c 1/m (Union bound). [ ] Pr max m2 j=1 Y j c 1 1/m (w.h.p.) E[max j Y j ] Ruta (UIUC) CS Spring / 32
71 Intuition: Throwing m Balls in to m 2 Bins Pr[ith ball lands in jth bin] = 1/m 2 For a fixed bin j, Y j =# balls in bin j. E[Y j ] = 1/m. For c 3, let δ = cm 1. Pr[Y j > c]? Pr[Y j > cm/m] = Pr[Y ( j > (1 ) + δ) E[Y j ]] µ e (Chernoff ) < δ (1+δ) ( (1+δ) ) 1/m e = (cm 1) (cm) (e/c) c (1/m c ) cm 1/m 3 [ ] Pr max m2 j=1 Y j > c 1/m (Union bound). [ ] Pr max m2 j=1 Y j c 1 1/m (w.h.p.) E[max j Y j ] c + 1 = O(1). Ruta (UIUC) CS Spring / 32
72 Perfect Hashing Two levels of hash tables Question: Can we make look up time O(1) in worst case? Perfect Hashing for Static Data Do hashing once. If Y i = l(i) > 10 then hash elements of l(i) to a table of size Y 2 i. Lemma (Look-up) Expected worst case look up time is O(1). Lemma (Size) If S = O(m) then space usage of perfect hashing is O(m). Ruta (UIUC) CS Spring / 32
73 Perfect Hashing: Proof of Lemma Size O(m) space usage h : the primary hash function. m i = # x in S such that h(x) = i. Ruta (UIUC) CS Spring / 32
74 Perfect Hashing: Proof of Lemma Size O(m) space usage h : the primary hash function. m i = # x in S such that h(x) = i. Ruta (UIUC) CS Spring / 32
75 Perfect Hashing: Proof of Lemma Size O(m) space usage h : the primary hash function. m i = # x in S such that h(x) = i. Claim E [ m 1 i =0 m2 i ] < 3m where m = S. Ruta (UIUC) CS Spring / 32
76 Perfect Hashing: Proof of Lemma Size O(m) space usage h : the primary hash function. m i = # x in S such that h(x) = i. Claim E [ m 1 i =0 m2 i ] < 3m where m = S. Proof. Let [h(x) = i ] represent indicator variable. m i = x S [h(x) = i ]. Ruta (UIUC) CS Spring / 32
77 Perfect Hashing: Proof of Lemma Size O(m) space usage h : the primary hash function. m i = # x in S such that h(x) = i. Claim E [ m 1 i =0 m2 i ] < 3m where m = S. Proof. Let [h(x) = i ] represent indicator variable. m i = x S [h(x) = i ]. i m2 i = i ( x S [h(x) = i ])2 Ruta (UIUC) CS Spring / 32
78 Perfect Hashing: Proof of Lemma Size O(m) space usage h : the primary hash function. m i = # x in S such that h(x) = i. Claim E [ m 1 i =0 m2 i ] < 3m where m = S. Proof. Let [h(x) = i ] represent indicator variable. m i = x S [h(x) = i ]. i m2 i = i ( x S [h(x) = i ])2 = i ( x [h(x) = i ]2 + 2 x<y [h(x) = i ][h(y) = i ]) Ruta (UIUC) CS Spring / 32
79 Perfect Hashing: Proof of Lemma Size O(m) space usage h : the primary hash function. m i = # x in S such that h(x) = i. Claim E [ m 1 i =0 m2 i ] < 3m where m = S. Proof. Let [h(x) = i ] represent indicator variable. m i = x S [h(x) = i ]. i m2 i = i ( x S [h(x) = i ])2 = i ( x [h(x) = i ]2 + 2 x<y = x ( i [h(x) = i ]) + 2 x<y [h(x) = i ][h(y) = i ]) [h(x) = i ][h(y) = i ] i Ruta (UIUC) CS Spring / 32
80 Perfect Hashing: Proof of Lemma Size O(m) space usage h : the primary hash function. m i = # x in S such that h(x) = i. Claim E [ m 1 i =0 m2 i ] < 3m where m = S. Proof. Let [h(x) = i ] represent indicator variable. m i = x S [h(x) = i ]. i m2 i = i ( x S [h(x) = i ])2 = i ( x [h(x) = i ]2 + 2 x<y = x ( i [h(x) = i ]) + 2 x<y = x (1) + 2 x<y [h(x) = h(y)] [h(x) = i ][h(y) = i ]) [h(x) = i ][h(y) = i ] i Ruta (UIUC) CS Spring / 32
81 Perfect Hashing: Proof of Lemma Size O(m) space usage h : the primary hash function. m i = # x in S such that h(x) = i. Claim E [ m 1 i =0 m2 i ] < 3m where m = S. Proof. Let [h(x) = i ] represent indicator variable. m i = x S [h(x) = i ]. i m2 i = i ( x S [h(x) = i ])2 = i ( x [h(x) = i ]2 + 2 x<y = x ( i [h(x) = i ]) + 2 x<y = x (1) + 2 x<y [h(x) = h(y)] E [ ] i m2 i = m + 2 x<y [h(x) = i ][h(y) = i ]) [h(x) = i ][h(y) = i ] i Pr[h(x) = h(y)] = m + 2 m(m 1) 2 1 m < 2m Ruta (UIUC) CS Spring / 32
82 Rehashing, amortization and making the hash table dynamic So far we assumed fixed S of size m. Question: What happens as items are inserted and deleted? 1 If S grows to more than cm for some constant c then hash table performance clearly degrades. 2 If S stays around m but incurs many insertions and deletions then the initial random hash function is no longer random enough! Ruta (UIUC) CS Spring / 32
83 Rehashing, amortization and making the hash table dynamic So far we assumed fixed S of size m. Question: What happens as items are inserted and deleted? 1 If S grows to more than cm for some constant c then hash table performance clearly degrades. 2 If S stays around m but incurs many insertions and deletions then the initial random hash function is no longer random enough! Solution: Rebuild hash table periodically! 1 Choose a new table size based on current number of elements in the table. 2 Choose a new random hash function and rehash the elements. 3 Discard old table and hash function. Question: When to rebuild? How expensive? Ruta (UIUC) CS Spring / 32
84 Rebuilding the hash table 1 Start with table size m where m is some estimate of S (can be some large constant). 2 If S grows to more than twice current table size, build new hash table (choose a new random hash function) with double the current number of elements. Can also use similar trick if table size falls below quarter the size. Ruta (UIUC) CS Spring / 32
85 Rebuilding the hash table 1 Start with table size m where m is some estimate of S (can be some large constant). 2 If S grows to more than twice current table size, build new hash table (choose a new random hash function) with double the current number of elements. Can also use similar trick if table size falls below quarter the size. 3 If S stays roughly the same but more than c S operations on table for some chosen constant c (say 10), rebuild. Ruta (UIUC) CS Spring / 32
86 Rebuilding the hash table 1 Start with table size m where m is some estimate of S (can be some large constant). 2 If S grows to more than twice current table size, build new hash table (choose a new random hash function) with double the current number of elements. Can also use similar trick if table size falls below quarter the size. 3 If S stays roughly the same but more than c S operations on table for some chosen constant c (say 10), rebuild. The amortize cost of rebuilding to previously performed operations. Rebuilding ensures O(1) expected analysis holds even when S changes. Hence O(1) expected look up/insert/delete time dynamic data dictionary data structure! Ruta (UIUC) CS Spring / 32
87 Bloom Filters Hashing: 1 To insert x in dictionary store x in table in location h(x) 2 To lookup y in dictionary check contents of location h(y) 3 Storing items in dictionary expensive in terms of memory, especially if items are unwieldy objects such a long strings, images, etc with non-uniform sizes. Ruta (UIUC) CS Spring / 32
88 Bloom Filters Hashing: 1 To insert x in dictionary store x in table in location h(x) 2 To lookup y in dictionary check contents of location h(y) 3 Storing items in dictionary expensive in terms of memory, especially if items are unwieldy objects such a long strings, images, etc with non-uniform sizes. Bloom Filter: tradeoff space for false positives 1 To insert x in dictionary set bit to 1 in location h(x) (initially all bits are set to 0) 2 To lookup y if bit in location h(y) is 1 say yes, else no. Ruta (UIUC) CS Spring / 32
89 Bloom Filters Bloom Filter: tradeoff space for false positives 1 To insert x in dictionary set bit to 1 in location h(x) (initially all bits are set to 0) 2 To lookup y if bit in location h(y) is 1 say yes, else no 3 No false negatives but false positives possible due to collisions Ruta (UIUC) CS Spring / 32
90 Bloom Filters Bloom Filter: tradeoff space for false positives 1 To insert x in dictionary set bit to 1 in location h(x) (initially all bits are set to 0) 2 To lookup y if bit in location h(y) is 1 say yes, else no 3 No false negatives but false positives possible due to collisions Reducing false positives: 1 Pick k hash functions h 1, h 2,..., h k independently Ruta (UIUC) CS Spring / 32
91 Bloom Filters Bloom Filter: tradeoff space for false positives 1 To insert x in dictionary set bit to 1 in location h(x) (initially all bits are set to 0) 2 To lookup y if bit in location h(y) is 1 say yes, else no 3 No false negatives but false positives possible due to collisions Reducing false positives: 1 Pick k hash functions h 1, h 2,..., h k independently 2 To insert set h i (x)th bit to one in table i for each 1 i k Ruta (UIUC) CS Spring / 32
92 Bloom Filters Bloom Filter: tradeoff space for false positives 1 To insert x in dictionary set bit to 1 in location h(x) (initially all bits are set to 0) 2 To lookup y if bit in location h(y) is 1 say yes, else no 3 No false negatives but false positives possible due to collisions Reducing false positives: 1 Pick k hash functions h 1, h 2,..., h k independently 2 To insert set h i (x)th bit to one in table i for each 1 i k 3 To lookup y compute h i (y) for 1 i k and say yes only if each bit in the corresponding location is 1, otherwise say no. If probability of false positive for one hash function is α < 1 then with k independent hash function it is α k. Ruta (UIUC) CS Spring / 32
93 Take away points 1 Hashing is a powerful and important technique for dictionaries. Many practical applications. 2 Randomization fundamental to understand hashing. 3 Good and efficient hashing possible in theory and practice with proper definitions (universal, perfect, etc). 4 Related ideas of creating a compact fingerprint/sketch for objects is very powerful in theory and practice. Ruta (UIUC) CS Spring / 32
94 Practical Issues Hashing used typically for integers, vectors, strings etc. Universal hashing is defined for integers. To implement for other objects need to map objects in some fashion to integers (via representation) Practical methods for various important cases such as vectors, strings are studied extensively. See for some pointers. Details on Cuckoo hashing and its advantage over chaining Relatively recent important paper bridging theory and practice of hashing. The power of simple tabulation hashing by Mikkel Thorup and Mihai Patrascu, See Cryptographic hash functions have a different motivation and requirements. Ruta (UIUC) Consequently CS473 they explore 32 different tradeoffs Spring 2018 and 32 / 32
6.1 Occupancy Problem
15-859(M): Randomized Algorithms Lecturer: Anupam Gupta Topic: Occupancy Problems and Hashing Date: Sep 9 Scribe: Runting Shi 6.1 Occupancy Problem Bins and Balls Throw n balls into n bins at random. 1.
Hash tables. Hash tables
Dictionary Definition A dictionary is a data-structure that stores a set of elements where each element has a unique key, and supports the following operations: Search(S, k) Return the element whose key
1 Maintaining a Dictionary
15-451/651: Design & Analysis of Algorithms February 1, 2016 Lecture #7: Hashing last changed: January 29, 2016 Hashing is a great practical tool, with an interesting and subtle theory too. In addition
Hash tables. Hash tables
Dictionary Definition A dictionary is a data-structure that stores a set of elements where each element has a unique key, and supports the following operations: Search(S, k) Return the element whose key
Lecture: Analysis of Algorithms (CS )
Lecture: Analysis of Algorithms (CS483-001) Amarda Shehu Spring 2017 1 Outline of Today s Class 2 Choosing Hash Functions Universal Universality Theorem Constructing a Set of Universal Hash Functions Perfect
1 Probability Review. CS 124 Section #8 Hashing, Skip Lists 3/20/17. Expectation (weighted average): the expectation of a random quantity X is:
CS 24 Section #8 Hashing, Skip Lists 3/20/7 Probability Review Expectation (weighted average): the expectation of a random quantity X is: x= x P (X = x) For each value x that X can take on, we look at
Analysis of Algorithms I: Perfect Hashing
Analysis of Algorithms I: Perfect Hashing Xi Chen Columbia University Goal: Let U = {0, 1,..., p 1} be a huge universe set. Given a static subset V U of n keys (here static means we will never change the
Hash tables. Hash tables
Basic Probability Theory Two events A, B are independent if Conditional probability: Pr[A B] = Pr[A] Pr[B] Pr[A B] = Pr[A B] Pr[B] The expectation of a (discrete) random variable X is E[X ] = k k Pr[X
Algorithms lecture notes 1. Hashing, and Universal Hash functions
Algorithms lecture notes 1 Hashing, and Universal Hash functions Algorithms lecture notes 2 Can we maintain a dictionary with O(1) per operation? Not in the deterministic sense. But in expectation, yes.
Motivation. Dictionaries. Direct Addressing. CSE 680 Prof. Roger Crawfis
Motivation Introduction to Algorithms Hash Tables CSE 680 Prof. Roger Crawfis Arrays provide an indirect way to access a set. Many times we need an association between two sets, or a set of keys and associated
Lecture 5: Hashing. David Woodruff Carnegie Mellon University
Lecture 5: Hashing David Woodruff Carnegie Mellon University Hashing Universal hashing Perfect hashing Maintaining a Dictionary Let U be a universe of keys U could be all strings of ASCII characters of
12 Hash Tables Introduction Chaining. Lecture 12: Hash Tables [Fa 10]
![12 Hash Tables Introduction Chaining. Lecture 12: Hash Tables [Fa 10]](/thumbs/82/85322860.jpg "12 Hash Tables Introduction Chaining. Lecture 12: Hash Tables [Fa 10]")
Calvin: There! I finished our secret code! Hobbes: Let s see. Calvin: I assigned each letter a totally random number, so the code will be hard to crack. For letter A, you write 3,004,572,688. B is 28,731,569½.
Hashing. Dictionaries Chained Hashing Universal Hashing Static Dictionaries and Perfect Hashing. Philip Bille
Hashing Dictionaries Chained Hashing Universal Hashing Static Dictionaries and Perfect Hashing Philip Bille Hashing Dictionaries Chained Hashing Universal Hashing Static Dictionaries and Perfect Hashing
Lecture 4 Thursday Sep 11, 2014
CS 224: Advanced Algorithms Fall 2014 Lecture 4 Thursday Sep 11, 2014 Prof. Jelani Nelson Scribe: Marco Gentili 1 Overview Today we re going to talk about: 1. linear probing (show with 5-wise independence)
Hashing. Hashing. Dictionaries. Dictionaries. Dictionaries Chained Hashing Universal Hashing Static Dictionaries and Perfect Hashing
Philip Bille Dictionaries Dictionary problem. Maintain a set S U = {,..., u-} supporting lookup(x): return true if x S and false otherwise. insert(x): set S = S {x} delete(x): set S = S - {x} Dictionaries
Algorithms for Data Science
Algorithms for Data Science CSOR W4246 Eleni Drinea Computer Science Department Columbia University Tuesday, December 1, 2015 Outline 1 Recap Balls and bins 2 On randomized algorithms 3 Saving space: hashing-based
CS5314 Randomized Algorithms. Lecture 15: Balls, Bins, Random Graphs (Hashing)
CS5314 Randomized Algorithms Lecture 15: Balls, Bins, Random Graphs (Hashing) 1 Objectives Study various hashing schemes Apply balls-and-bins model to analyze their performances 2 Chain Hashing Suppose
Hash Tables. Given a set of possible keys U, such that U = u and a table of m entries, a Hash function h is a
Hash Tables Given a set of possible keys U, such that U = u and a table of m entries, a Hash function h is a mapping from U to M = {1,..., m}. A collision occurs when two hashed elements have h(x) =h(y).
compare to comparison and pointer based sorting, binary trees
Admin Hashing Dictionaries Model Operations. makeset, insert, delete, find keys are integers in M = {1,..., m} (so assume machine word size, or unit time, is log m) can store in array of size M using power:
CSE 190, Great ideas in algorithms: Pairwise independent hash functions
CSE 190, Great ideas in algorithms: Pairwise independent hash functions 1 Hash functions The goal of hash functions is to map elements from a large domain to a small one. Typically, to obtain the required
Problem 1: (Chernoff Bounds via Negative Dependence - from MU Ex 5.15)
Problem 1: Chernoff Bounds via Negative Dependence - from MU Ex 5.15) While deriving lower bounds on the load of the maximum loaded bin when n balls are thrown in n bins, we saw the use of negative dependence.
CSCB63 Winter Week10 - Lecture 2 - Hashing. Anna Bretscher. March 21, / 30
CSCB63 Winter 2019 Week10 - Lecture 2 - Hashing Anna Bretscher March 21, 2019 1 / 30 Today Hashing Open Addressing Hash functions Universal Hashing 2 / 30 Open Addressing Open Addressing. Each entry in
CS 591, Lecture 6 Data Analytics: Theory and Applications Boston University
CS 591, Lecture 6 Data Analytics: Theory and Applications Boston University Babis Tsourakakis February 8th, 2017 Universal hash family Notation: Universe U = {0,..., u 1}, index space M = {0,..., m 1},
So far we have implemented the search for a key by carefully choosing split-elements.
7.7 Hashing Dictionary: S. insert(x): Insert an element x. S. delete(x): Delete the element pointed to by x. S. search(k): Return a pointer to an element e with key[e] = k in S if it exists; otherwise
Lecture Lecture 3 Tuesday Sep 09, 2014
CS 4: Advanced Algorithms Fall 04 Lecture Lecture 3 Tuesday Sep 09, 04 Prof. Jelani Nelson Scribe: Thibaut Horel Overview In the previous lecture we finished covering data structures for the predecessor
CS483 Design and Analysis of Algorithms
CS483 Design and Analysis of Algorithms Lectures 2-3 Algorithms with Numbers Instructor: Fei Li lifei@cs.gmu.edu with subject: CS483 Office hours: STII, Room 443, Friday 4:00pm - 6:00pm or by appointments
Cache-Oblivious Hashing
Cache-Oblivious Hashing Zhewei Wei Hong Kong University of Science & Technology Joint work with Rasmus Pagh, Ke Yi and Qin Zhang Dictionary Problem Store a subset S of the Universe U. Lookup: Does x belong
As mentioned, we will relax the conditions of our dictionary data structure. The relaxations we
CSE 203A: Advanced Algorithms Prof. Daniel Kane Lecture : Dictionary Data Structures and Load Balancing Lecture Date: 10/27 P Chitimireddi Recap This lecture continues the discussion of dictionary data
Lecture and notes by: Alessio Guerrieri and Wei Jin Bloom filters and Hashing
Bloom filters and Hashing 1 Introduction The Bloom filter, conceived by Burton H. Bloom in 1970, is a space-efficient probabilistic data structure that is used to test whether an element is a member of
Hashing. Martin Babka. January 12, 2011
Hashing Martin Babka January 12, 2011 Hashing Hashing, Universal hashing, Perfect hashing Input data is uniformly distributed. A dynamic set is stored. Universal hashing Randomised algorithm uniform choice
Symbol-table problem. Hashing. Direct-access table. Hash functions. CS Spring Symbol table T holding n records: record.
CS 5633 -- Spring 25 Symbol-table problem Hashing Carola Wenk Slides courtesy of Charles Leiserson with small changes by Carola Wenk CS 5633 Analysis of Algorithms 1 Symbol table holding n records: record
1 Estimating Frequency Moments in Streams
CS 598CSC: Algorithms for Big Data Lecture date: August 28, 2014 Instructor: Chandra Chekuri Scribe: Chandra Chekuri 1 Estimating Frequency Moments in Streams A significant fraction of streaming literature
Application: Bucket Sort
5.2.2. Application: Bucket Sort Bucket sort breaks the log) lower bound for standard comparison-based sorting, under certain assumptions on the input We want to sort a set of =2 integers chosen I+U@R from
CS369N: Beyond Worst-Case Analysis Lecture #6: Pseudorandom Data and Universal Hashing
CS369N: Beyond Worst-Case Analysis Lecture #6: Pseudorandom Data and Universal Hashing Tim Roughgarden April 4, 204 otivation: Linear Probing and Universal Hashing This lecture discusses a very neat paper
? 11.5 Perfect hashing. Exercises
11.5 Perfect hashing 77 Exercises 11.4-1 Consider inserting the keys 10; ; 31; 4; 15; 8; 17; 88; 59 into a hash table of length m 11 using open addressing with the auxiliary hash function h 0.k/ k. Illustrate
Introduction to Hash Tables
Introduction to Hash Tables Hash Functions A hash table represents a simple but efficient way of storing, finding, and removing elements. In general, a hash table is represented by an array of cells. In
Basics of hashing: k-independence and applications
Basics of hashing: k-independence and applications Rasmus Pagh Supported by: 1 Agenda Load balancing using hashing! - Analysis using bounded independence! Implementation of small independence! Case studies:!
Cuckoo Hashing with a Stash: Alternative Analysis, Simple Hash Functions
1 / 29 Cuckoo Hashing with a Stash: Alternative Analysis, Simple Hash Functions Martin Aumüller, Martin Dietzfelbinger Technische Universität Ilmenau 2 / 29 Cuckoo Hashing Maintain a dynamic dictionary
2. This exam consists of 15 questions. The rst nine questions are multiple choice Q10 requires two
CS{74 Combinatorics & Discrete Probability, Fall 96 Final Examination 2:30{3:30pm, 7 December Read these instructions carefully. This is a closed book exam. Calculators are permitted. 2. This exam consists
Abstract Data Type (ADT) maintains a set of items, each with a key, subject to
Lecture Overview Dictionaries and Python Motivation Hash functions Chaining Simple uniform hashing Good hash functions Readings CLRS Chapter,, 3 Dictionary Problem Abstract Data Type (ADT) maintains a
Advanced Data Structures
Simon Gog gog@kit.edu - 0 Simon Gog: KIT University of the State of Baden-Wuerttemberg and National Research Center of the Helmholtz Association www.kit.edu Dynamic Perfect Hashing What we want: O(1) lookup
Aside: Golden Ratio. Golden Ratio: A universal law. Golden ratio φ = lim n = 1+ b n = a n 1. a n+1 = a n + b n, a n+b n a n
Aside: Golden Ratio Golden Ratio: A universal law. Golden ratio φ = lim n a n+b n a n = 1+ 5 2 a n+1 = a n + b n, b n = a n 1 Ruta (UIUC) CS473 1 Spring 2018 1 / 41 CS 473: Algorithms, Spring 2018 Dynamic
Lecture 4 February 2nd, 2017
CS 224: Advanced Algorithms Spring 2017 Prof. Jelani Nelson Lecture 4 February 2nd, 2017 Scribe: Rohil Prasad 1 Overview In the last lecture we covered topics in hashing, including load balancing, k-wise
Introduction to Hashtables
Introduction to HashTables Boise State University March 5th 2015 Hash Tables: What Problem Do They Solve What Problem Do They Solve? Why not use arrays for everything? 1 Arrays can be very wasteful: Example
Tabulation-Based Hashing
Mihai Pǎtraşcu Mikkel Thorup April 23, 2010 Applications of Hashing Hash tables: chaining x a t v f s r Applications of Hashing Hash tables: chaining linear probing m c a f t x y t Applications of Hashing
More Approximation Algorithms
CS 473: Algorithms, Spring 2018 More Approximation Algorithms Lecture 25 April 26, 2018 Most slides are courtesy Prof. Chekuri Ruta (UIUC) CS473 1 Spring 2018 1 / 28 Formal definition of approximation
CSCB63 Winter Week 11 Bloom Filters. Anna Bretscher. March 30, / 13
CSCB63 Winter 2019 Week 11 Bloom Filters Anna Bretscher March 30, 2019 1 / 13 Today Bloom Filters Definition Expected Complexity Applications 2 / 13 Bloom Filters (Specification) A bloom filter is a probabilistic
Data Structure. Mohsen Arab. January 13, Yazd University. Mohsen Arab (Yazd University ) Data Structure January 13, / 86
Data Structure Mohsen Arab Yazd University January 13, 2015 Mohsen Arab (Yazd University ) Data Structure January 13, 2015 1 / 86 Table of Content Binary Search Tree Treaps Skip Lists Hash Tables Mohsen
Lecture 8 HASHING!!!!!
Lecture 8 HASHING!!!!! Announcements HW3 due Friday! HW4 posted Friday! Q: Where can I see examples of proofs? Lecture Notes CLRS HW Solutions Office hours: lines are long L Solutions: We will be (more)
CS 580: Algorithm Design and Analysis
CS 580: Algorithm Design and Analysis Jeremiah Blocki Purdue University Spring 2018 Announcements: Homework 6 deadline extended to April 24 th at 11:59 PM Course Evaluation Survey: Live until 4/29/2018
Collision. Kuan-Yu Chen ( 陳冠宇 ) TR-212, NTUST
Collision Kuan-Yu Chen ( 陳冠宇 ) 2018/12/17 @ TR-212, NTUST Review Hash table is a data structure in which keys are mapped to array positions by a hash function When two or more keys map to the same memory
Some notes on streaming algorithms continued
U.C. Berkeley CS170: Algorithms Handout LN-11-9 Christos Papadimitriou & Luca Trevisan November 9, 016 Some notes on streaming algorithms continued Today we complete our quick review of streaming algorithms.
Hashing, Hash Functions. Lecture 7
Hashing, Hash Functions Lecture 7 Symbol-table problem Symbol table T holding n records: x record key[x] Other fields containing satellite data Operations on T: INSERT(T, x) DELETE(T, x) SEARCH(T, k) How
1 Difference between grad and undergrad algorithms
princeton univ. F 4 cos 52: Advanced Algorithm Design Lecture : Course Intro and Hashing Lecturer: Sanjeev Arora Scribe:Sanjeev Algorithms are integral to computer science and every computer scientist
15-451/651: Design & Analysis of Algorithms September 13, 2018 Lecture #6: Streaming Algorithms last changed: August 30, 2018
15-451/651: Design & Analysis of Algorithms September 13, 2018 Lecture #6: Streaming Algorithms last changed: August 30, 2018 Today we ll talk about a topic that is both very old (as far as computer science
Outline. CS38 Introduction to Algorithms. Max-3-SAT approximation algorithm 6/3/2014. randomness in algorithms. Lecture 19 June 3, 2014
Outline CS38 Introduction to Algorithms Lecture 19 June 3, 2014 randomness in algorithms max-3-sat approximation algorithm universal hashing load balancing Course summary and review June 3, 2014 CS38 Lecture
Cuckoo Hashing and Cuckoo Filters
Cuckoo Hashing and Cuckoo Filters Noah Fleming May 7, 208 Preliminaries A dictionary is an abstract data type that stores a collection of elements, located by their key. It supports operations: insert,
Introduction to discrete probability. The rules Sample space (finite except for one example)
Algorithms lecture notes 1 Introduction to discrete probability The rules Sample space (finite except for one example) say Ω. P (Ω) = 1, P ( ) = 0. If the items in the sample space are {x 1,..., x n }
14.1 Finding frequent elements in stream
Chapter 14 Streaming Data Model 14.1 Finding frequent elements in stream A very useful statistics for many applications is to keep track of elements that occur more frequently. It can come in many flavours
1 Hashing. 1.1 Perfect Hashing
1 Hashing Hashing is covered by undergraduate courses like Algo I. However, there is much more to say on this topic. Here, we focus on two selected topics: perfect hashing and cockoo hashing. In general,
Searching. Constant time access. Hash function. Use an array? Better hash function? Hash function 4/18/2013. Chapter 9
Constant time access Searching Chapter 9 Linear search Θ(n) OK Binary search Θ(log n) Better Can we achieve Θ(1) search time? CPTR 318 1 2 Use an array? Use random access on a key such as a string? Hash
Insert Sorted List Insert as the Last element (the First element?) Delete Chaining. 2 Slide courtesy of Dr. Sang-Eon Park
1617 Preview Data Structure Review COSC COSC Data Structure Review Linked Lists Stacks Queues Linked Lists Singly Linked List Doubly Linked List Typical Functions s Hash Functions Collision Resolution
Hashing. Dictionaries Hashing with chaining Hash functions Linear Probing
Hashing Dictionaries Hashing with chaining Hash functions Linear Probing Hashing Dictionaries Hashing with chaining Hash functions Linear Probing Dictionaries Dictionary: Maintain a dynamic set S. Every
Problem. Maintain the above set so as to support certain
Randomized Data Structures Roussos Ioannis Fundamental Data-structuring Problem Collection {S 1,S 2, } of sets of items. Each item i is an arbitrary record, indexed by a key k(i). Each k(i) is drawn from
CSE 548: (Design and) Analysis of Algorithms
Intro Decentralize Taming distribution Probabilistic Algorithms 1 / 68 CSE 548: (Design and) Analysis of Algorithms Randomized Algorithms R. Sekar Intro Decentralize Taming distribution Probabilistic Algorithms
Lecture 12: Hash Tables [Sp 15]
![Lecture 12: Hash Tables [Sp 15]](/thumbs/82/85614941.jpg "Lecture 12: Hash Tables [Sp 15]")
Insanity is repeating the same mistaes and expecting different results. Narcotics Anonymous (1981) Calvin: There! I finished our secret code! Hobbes: Let s see. Calvin: I assigned each letter a totally
Proving languages to be nonregular
Proving languages to be nonregular We already know that there exist languages A Σ that are nonregular, for any choice of an alphabet Σ. This is because there are uncountably many languages in total and
Data Structures and Algorithm. Xiaoqing Zheng
Data Structures and Algorithm Xiaoqing Zheng zhengxq@fudan.edu.cn Dictionary problem Dictionary T holding n records: x records key[x] Other fields containing satellite data Operations on T: INSERT(T, x)
Lecture Note 2. 1 Bonferroni Principle. 1.1 Idea. 1.2 Want. Material covered today is from Chapter 1 and chapter 4
Lecture Note 2 Material covere toay is from Chapter an chapter 4 Bonferroni Principle. Iea Get an iea the frequency of events when things are ranom billion = 0 9 Each person has a % chance to stay in a
Advanced Algorithm Design: Hashing and Applications to Compact Data Representation
Advanced Algorithm Design: Hashing and Applications to Compact Data Representation Lectured by Prof. Moses Chariar Transcribed by John McSpedon Feb th, 20 Cucoo Hashing Recall from last lecture the dictionary
Hashing Data Structures. Ananda Gunawardena
Hashing 15-121 Data Structures Ananda Gunawardena Hashing Why do we need hashing? Many applications deal with lots of data Search engines and web pages There are myriad look ups. The look ups are time
Explicit and Efficient Hash Functions Suffice for Cuckoo Hashing with a Stash
Explicit and Efficient Hash Functions Suffice for Cuckoo Hashing with a Stash Martin Aumu ller, Martin Dietzfelbinger, Philipp Woelfel? Technische Universita t Ilmenau,? University of Calgary ESA Ljubljana,
arxiv: v1 [cs.ds] 3 Feb 2018
![arxiv: v1 [cs.ds] 3 Feb 2018](/thumbs/92/108786758.jpg "arxiv: v1 [cs.ds] 3 Feb 2018")
A Model for Learned Bloom Filters and Related Structures Michael Mitzenmacher 1 arxiv:1802.00884v1 [cs.ds] 3 Feb 2018 Abstract Recent work has suggested enhancing Bloom filters by using a pre-filter, based
A Lecture on Hashing. Aram-Alexandre Pooladian, Alexander Iannantuono March 22, Hashing. Direct Addressing. Operations - Simple
A Lecture on Hashing Aram-Alexandre Pooladian, Alexander Iannantuono March 22, 217 This is the scribing of a lecture given by Luc Devroye on the 17th of March 217 for Honours Algorithms and Data Structures
Searching, mainly via Hash tables
Data structures and algorithms Part 11 Searching, mainly via Hash tables Petr Felkel 26.1.2007 Topics Searching Hashing Hash function Resolving collisions Hashing with chaining Open addressing Linear Probing
CPSC 467: Cryptography and Computer Security
CPSC 467: Cryptography and Computer Security Michael J. Fischer Lecture 14 October 16, 2013 CPSC 467, Lecture 14 1/45 Message Digest / Cryptographic Hash Functions Hash Function Constructions Extending
Introduction to Randomized Algorithms III
Introduction to Randomized Algorithms III Joaquim Madeira Version 0.1 November 2017 U. Aveiro, November 2017 1 Overview Probabilistic counters Counting with probability 1 / 2 Counting with probability
A Model for Learned Bloom Filters, and Optimizing by Sandwiching
A Model for Learned Bloom Filters, and Optimizing by Sandwiching Michael Mitzenmacher School of Engineering and Applied Sciences Harvard University michaelm@eecs.harvard.edu Abstract Recent work has suggested
CSE 502 Class 11 Part 2
CSE 502 Class 11 Part 2 Jeremy Buhler Steve Cole February 17 2015 Today: analysis of hashing 1 Constraints of Double Hashing How does using OA w/double hashing constrain our hash function design? Need
Tight Bounds for Sliding Bloom Filters
Tight Bounds for Sliding Bloom Filters Moni Naor Eylon Yogev November 7, 2013 Abstract A Bloom filter is a method for reducing the space (memory) required for representing a set by allowing a small error
The first bound is the strongest, the other two bounds are often easier to state and compute. Proof: Applying Markov's inequality, for any >0 we have
The first bound is the strongest, the other two bounds are often easier to state and compute Proof: Applying Markov's inequality, for any >0 we have Pr (1 + ) = Pr For any >0, we can set = ln 1+ (4.4.1):
B490 Mining the Big Data
B490 Mining the Big Data 1 Finding Similar Items Qin Zhang 1-1 Motivations Finding similar documents/webpages/images (Approximate) mirror sites. Application: Don t want to show both when Google. 2-1 Motivations
s 1 if xπy and f(x) = f(y)
Algorithms Proessor John Rei Hash Function : A B ALG 4.2 Universal Hash Functions: CLR - Chapter 34 Auxillary Reading Selections: AHU-Data Section 4.7 BB Section 8.4.4 Handout: Carter & Wegman, "Universal
Lecture 4: Two-point Sampling, Coupon Collector s problem
Randomized Algorithms Lecture 4: Two-point Sampling, Coupon Collector s problem Sotiris Nikoletseas Associate Professor CEID - ETY Course 2013-2014 Sotiris Nikoletseas, Associate Professor Randomized Algorithms
Lecture 2. Frequency problems
1 / 43 Lecture 2. Frequency problems Ricard Gavaldà MIRI Seminar on Data Streams, Spring 2015 Contents 2 / 43 1 Frequency problems in data streams 2 Approximating inner product 3 Computing frequency moments
CS 591, Lecture 7 Data Analytics: Theory and Applications Boston University
CS 591, Lecture 7 Data Analytics: Theory and Applications Boston University Babis Tsourakakis February 13th, 2017 Bloom Filter Approximate membership problem Highly space-efficient randomized data structure
Bloom Filters, general theory and variants
Bloom Filters: general theory and variants G. Caravagna caravagn@cli.di.unipi.it Information Retrieval Wherever a list or set is used, and space is a consideration, a Bloom Filter should be considered.
Discrete Mathematics and Probability Theory Fall 2014 Anant Sahai Note 15. Random Variables: Distributions, Independence, and Expectations
EECS 70 Discrete Mathematics and Probability Theory Fall 204 Anant Sahai Note 5 Random Variables: Distributions, Independence, and Expectations In the last note, we saw how useful it is to have a way of
Bloom Filters, Adaptivity, and the Dictionary Problem
2018 IEEE 59th Annual Symposium on Foundations of Computer Science Bloom Filters, Adaptivity, and the Dictionary Problem Michael A. Bender, Martín Farach-Colton, Mayank Goswami, Rob Johnson, Samuel McCauley,
COMPUTING SIMILARITY BETWEEN DOCUMENTS (OR ITEMS) This part is to a large extent based on slides obtained from
COMPUTING SIMILARITY BETWEEN DOCUMENTS (OR ITEMS) This part is to a large extent based on slides obtained from http://www.mmds.org Distance Measures For finding similar documents, we consider the Jaccard
1 Randomized Computation
CS 6743 Lecture 17 1 Fall 2007 1 Randomized Computation Why is randomness useful? Imagine you have a stack of bank notes, with very few counterfeit ones. You want to choose a genuine bank note to pay at
Advanced Implementations of Tables: Balanced Search Trees and Hashing
Advanced Implementations of Tables: Balanced Search Trees and Hashing Balanced Search Trees Binary search tree operations such as insert, delete, retrieve, etc. depend on the length of the path to the
CS261: A Second Course in Algorithms Lecture #18: Five Essential Tools for the Analysis of Randomized Algorithms
CS261: A Second Course in Algorithms Lecture #18: Five Essential Tools for the Analysis of Randomized Algorithms Tim Roughgarden March 3, 2016 1 Preamble In CS109 and CS161, you learned some tricks of
Randomized Algorithms. Zhou Jun
Randomized Algorithms Zhou Jun 1 Content 13.1 Contention Resolution 13.2 Global Minimum Cut 13.3 *Random Variables and Expectation 13.4 Randomized Approximation Algorithm for MAX 3- SAT 13.6 Hashing 13.7
Discrete Mathematics for CS Spring 2006 Vazirani Lecture 22
CS 70 Discrete Mathematics for CS Spring 2006 Vazirani Lecture 22 Random Variables and Expectation Question: The homeworks of 20 students are collected in, randomly shuffled and returned to the students.
Lecture 24: Approximate Counting
CS 710: Complexity Theory 12/1/2011 Lecture 24: Approximate Counting Instructor: Dieter van Melkebeek Scribe: David Guild and Gautam Prakriya Last time we introduced counting problems and defined the class
6.842 Randomness and Computation Lecture 5
6.842 Randomness and Computation 2012-02-22 Lecture 5 Lecturer: Ronitt Rubinfeld Scribe: Michael Forbes 1 Overview Today we will define the notion of a pairwise independent hash function, and discuss its
Lecture 4: Hashing and Streaming Algorithms
CSE 521: Design and Analysis of Algorithms I Winter 2017 Lecture 4: Hashing and Streaming Algorithms Lecturer: Shayan Oveis Gharan 01/18/2017 Scribe: Yuqing Ai Disclaimer: These notes have not been subjected
More NP-Complete Problems
CS 473: Algorithms, Spring 2018 More NP-Complete Problems Lecture 23 April 17, 2018 Most slides are courtesy Prof. Chekuri Ruta (UIUC) CS473 1 Spring 2018 1 / 57 Recap NP: languages/problems that have
Piazza Recitation session: Review of linear algebra Location: Thursday, April 11, from 3:30-5:20 pm in SIG 134 (here)
4/0/9 Tim Althoff, UW CS547: Machine Learning for Big Data, http://www.cs.washington.edu/cse547 Piazza Recitation session: Review of linear algebra Location: Thursday, April, from 3:30-5:20 pm in SIG 34