Algorithms lecture notes 1. Hashing, and Universal Hash functions

Transcription

1 Algorithms lecture notes 1 Hashing, and Universal Hash functions

2 Algorithms lecture notes 2 Can we maintain a dictionary with O(1) per operation? Not in the deterministic sense. But in expectation, yes. Say that we try to use a balanced binary search tree. We can do n inserts at time Θ(logn) per item. This takes Θ(nlogn). And then we can run Inorder. This implies that we can not maintain a balanced tree with O(1) per insert. Because this will mean O(n) time for sorting.

3 Algorithms lecture notes 3 Universe U. U = n. Hashing And an array A[0,1,...,n 1]. Put x U in some place h(x) according to some function h. First idea: Throw x into a random entry of A. Best behavior. But given x we have no idea were it went. Collisions: h(x) = h(y) and x y. If we get collision we start a linked list at entry h(x). If all numbers are hashed to the same entry, all we get is a linked list, and we gained nothing. We want h to be as close to random as possible

4 Algorithms lecture notes 4 H($))=0, H(21)=1, H(6)=1, H(33)=1, H(50)=6 H(27)=4, H(70)=4, H[42]= NILL NILL 2 NILL 3 NILL NILL 5 NILL 6 50 NILL 7 NILL Figure 1: A Hash function with collisions

5 Algorithms lecture notes 5 Examples: A very bad idea h(k) = k. Even if you have 10 numbers you need a huge table if one of the x is large. Division method: the universe has integers. h(k) = k mod n. k can be much larger than n. A very bad example is n = 2 r. Say k = 2 p with p >> r n = 2 5. h depends only on first 5 bits. Do not pick m that has a small divisor d. If i = j mod d then i mod n = i mod d q h(i) mod d q = i mod d i mod q. h(j) mod d q = j mod d i mod q. Thus we went from m to q. Example: say i = and we do n = 100. Then h(i) = 1.

6 Algorithms lecture notes 6 Making n numbers collide 1+n,1+2n,1+3n,...1+n 2. All go to A[1] after mod n. As bad as a linked list. Another example: using multiplication. 1. Choose some rational number A (not integer). 2. Compute A i. 3. Let f(a i) be the numbers after the dot. 4. H(i) = f n Goes to 0,1,...,n 1. And now to the bad news.

7 Algorithms lecture notes 7 Bad input for any hash function Let U = n 2. Let h be any has function. Compute h(u) for any u U. This gives n 2 values. Each value h(u) {0,1,2,...,n 1}. Let G(i) be those who get to i. n 1 i=0 G(i) = n2. The average is n 2 /n = n. Thus there some G(i) n. This means that there are n numbers that all go to the same entry i. All (deterministic) hash function can be chosen an input that will give a linked list.

8 Algorithms lecture notes 8 Perfect Hash function: a genius idea We are going to have a hash function chosen at random from a set of hash functions. 4054,1 95Say H = {h 1,...,h r } we will choose some h i so each i is equally likely. We want it to look random after we do this. What does it mean look random. Let x,y be two different keys in the universe. Will x,y collide? This depends on the question if h i makes them collide, or in other words, on the fraction of {h i } that make them collide. Say that q of the h i make x,y collide. Then P(x,y collide) = q/r. Definition: A collection of hash functions is universal if for every x,y q/r 1/n. This may seem impossible.

9 Algorithms lecture notes 9 Universal hash function We want that for every x,y that if q is the number of hash factions that make x,y collide then q/r 1/n. This basically gives a random behavior. Let us compute the number of elements that will arrive to slot i. Say that x arrives to slot i. Let Z x,y = 1 if x,y collide and 0 otherwise. The number of elements in i is 1+ y x Z x,y. By the universality of the function the expected number of entries in slot i is simply 1+(n 1)/n < 2. Very good behavior.

10 Algorithms lecture notes 10 Designing a universal hash function Things to know from number theory. Let p be a prime and look at Z p = {0,1,2,...,p 1}. Clearly every number i has a unique j so that i+j = 0 mod p. Denote Z p = {1,2,...,p 1}. Its known that every number here has a unique inverse for a prime p. Example p = 7 1 is its own inverse. 2 and 4 are inverses. 3 and 5 are inverses. Note that p 1 is always its own inverse: (p 1) 2 = p 2 2p+1 = 1 mod p. So 6 is its own inverse. Let p be a prime number much larger than n. Intuitively, given any i,j U U we first want to replace them by 2 random different numbers from Z p.

11 Algorithms lecture notes 11 Turning i,j into two random different numbers Choose a from {1,2,...,p 1} and b from {0,1,2...,p 1}. The hash functions are all a i+b mod p a 0. Thus p (p 1) hash functions. Each equally likely to be chosen. The two numbers are different as otherwise a i = a j mod p. Multiply by a 1, i = j mod p contradiction.

12 Algorithms lecture notes 12 Why are they random distinct numbers? Choose r from Z p at random and s at random from Z p \{r}. Note that for both a i+b mod p = r to hold and a j +b mod p = s to hold, we need a(i j) = (r s) mod p. Or a = (i j) 1 (r s) mod p. This has probability 1/(p 1). And b = (r a i) mod p has probability 1/p as a i is a random non zero number in Z p. Thus we get 1/(p (p 1)) probability for every distinct s,r. Thus s,r are two distinct random numbers.

13 Algorithms lecture notes 13 Taking mod n after the modulo p The function is ((ax+b) mod p) mod n. If r and s are chosen at random from all p (p 1) distinct pairs of numbers in Z p, what is the probability that they collide module m? Say that we choose r first and then choose s Z p \{r}. Out of the p that belong to Z p which are numbers that will cause a collision when we go modulo n? Numbers of Z p that will cause a collision: s = r +n, s = r +2n, s = r+3n... s = r +( p/n 1) n. These are the choices for s that collide with r.

14 Algorithms lecture notes 14 Collisions These are not more than (p 1)/n numbers. s is different than r and is chosen at random from {0,1,...,p 1}\{r}. The probability for s to be one of those is not larger than (p 1)/n p 1 = 1/n. As we wanted.

15 Algorithms lecture notes 15 A dictionary The expected size of every entry is constant. Thus in expectation the operation Insert(S, x) Search(S, x) and Delete(S, x) require O(1) expected number of operations. This is expectation. What can we say with high probability on the number of items that reach slot i? With probability at least 1 1/2 b, the number of items This is too advanced to be shown here. Still its less than Ω(logn) even if we want it to be true with probability very close to 1.

16 Algorithms lecture notes 16 Are there two numbers in an array that sum to S We use a universal Hash function. For every x we look for S x in the table. For every x the expected search time is O(1) Therefore we get that we can check the above with expected time O(n) Recall that for a deterministic algorithm the best time we had was worse: Θ(nlogn).

17 Algorithms lecture notes 16-1