1 Hash tables. 1.1 Implementation

Transcription

1 Lecture 8 Hash Tables, Uiversal Hash Fuctios, Balls ad Bis Scribes: Luke Johsto, Moses Charikar, G. Valiat Date: Oct 18, 2017 Adapted From Virgiia Williams lecture otes 1 Hash tables A hash table is a commoly used data structure to store a uordered set of items, allowig costat time iserts, lookups ad deletes (i expectatio). Every item cosists of a uique idetifier called a key ad a piece of iformatio. For example, the key might be a Social Security Number, a driver s licese umber, or a employee ID umber. The way i which a hash table stores a item depeds oly o its key, so we will oly focus o the key here, but keep i mid that each key is usually associated with additioal iformatio that is also stored i the hash table. A hash table supports the followig operatios: Isert(k): Isert key k ito the hash table. Lookup(k): Check if key k is preset i the table. Delete(k): Delete the key k from the table. Each operatio will take costat time (i expectatio). 1.1 Implemetatio Let U be the uiverse of all keys. For example, U could be the set of all 64 bit strigs. I this case U = This is a very large uiverse, but we do ot eed to store all of these 2 64 keys, we oly eed to store a subset S U. Suppose that we kow that the size of the subset we will eed to store is less tha or equal to, which is much less tha the size of the uiverse U. I a hash table of size, each key k U is mapped to oe of buckets by a hash fuctio h : U {1, 2,..., }. Sice the uiverse U is much larger tha, multiple keys could map to the same hash bucket. To accommodate this, each bucket cotais a liked list of keys curretly stored i that bucket. Example Suppose we have a hash table of size = 5 with hash fuctio h(x) = 13x + 2 mod 5. After isertig the elemets {1, 2, 4, 7, 8} the hash table looks like this: B 0 1 B 1 8 B 2 NIL B B 4 4 Where arrows deote poiters i the liked lists, ad B 2 is empty. For example, 1 is placed ito bucket B 0 because h(1) = 15 mod 5 = 0. 1

2 Time complexity With this setup, the time required to perform a Isert, Lookup or Delete operatio o key k is liear i the legth of the liked list for the bucket that key k maps to. We just use the hash fuctio to fid the correct bucket for a iput key k, ad the search the correspodig liked list for the elemet, isertig or deletig if ecessary. (Note that a Isert could be performed i costat time by always isertig at the head of the list, but we first eed to check if key k is already preset). Choice of size of hash table The hash table size is usually chose so that the size of the hash table is at least as large as the maximum umber of keys we will eed to store at ay poit of time. (If this coditio is violated ad the umber of keys stored grows much larger tha the size of the hash table, a implemetatio will usually icrease the size of the table, ad recompute the ew table from scratch by mappig all keys to the bigger table. Our aalyis igores these complicatios ad assumes that the umber of keys is at most the hash table size.) Potetial problem with this implemetatio I order for the operatios to be implemeted efficietly, we would like the keys to be distributed uiformly amogst the buckets i the hash table. We might hope that all buckets have at most a costat umber of keys mapped to them, so that all operatios could be performed i costat time. But for ay fixed choice hash fuctio h, oe ca always produce a subset of keys S such that all keys i S are mapped to the same locatio i the hash table. I this case, the ruig times of all operatios will be liear i the umber of keys far from the costat we were hopig for. Thus, for a fixed hash fuctio h, it is impossible to give worst case guaratees for the ruig times of hash table operatios. Possible solutios There are two styles of aalysis that we could use to circumvet this problem: 1. Assume that the set of keys stored i the hash table is radom, or 2. Assume that the hash fuctio h is radom. Both are plausible alteratives. The problem with the first alterative is that it is hard to justify that the set of keys stored i the hash table is truly radom. It would be more satisfyig to have a aalysis that works for ay subset of keys curretly i the hash table. I these otes, we will explore the secod alterative, i.e., assume that the hash fuctio h is radom. 1.2 Hashig with a completely radom hash fuctio What does it mea for h to be radom? Oe possibility is that h is chose uiformly ad at radom from amogst the set of all hash fuctios h : U {1, 2,..., }. I fact pickig such a hash fuctio is ot really practical. Note that there are U possible hash fuctios. Represetig just oe of these hash fuctios requires log ( U ) = U log bits. I fact, this meas we eed to write dow h(x) for every x U i order to represet h. That s a lot of storage space! Much more tha the size of the set we are tryig to store i the hash table. Oe could optimize this somewhat by oly recordig h(x) for all keys x see so far (ad geeratig h(x) radomly o the fly whe a ew x is ecoutered), but this is impractical too. How would we check if a particular key x has already bee ecoutered? Looks like we would eed a hash table for that. But wait, is t that what we set out to implemet? Overall, it is clear that pickig a completely radom hash fuctio is completely impractical. Despite this, we will aalyze hashig assumig that we have a completely radom hash fuctio ad the explai how this assumptio ca be replaced by somethig that is practical. 2

3 Expected cost of hash table operatios with radom hash fuctio What is the expected cost of performig ay of the operatios Isert, Lookup or Delete with a radom hash fuctio? Suppose that the keys curretly i the hash table are k 1,..., k. Cosider a operatio ivolvig key k i. The cost of the operatio is liear i the size of the hash bucket that k i maps to. Let X be the size of the hash bucket that k i maps to. X is a radom variable ad E[X] = Pr[h(x i ) = h(x j )] j=1 = 1 + j i Pr[h(x i ) = h(x j )] = Here the last step follows from the fact that Pr[h(x i ) = h(x j )] = 1/ whe h is radom. Note that each key appears i the hash table at most oce. Thus the expected cost of ay hashig operatio is a costat. 1.3 Uiversal hash fuctios Ca we retai the expected cost guaratee of the previous sectio with a much simpler (i.e. practical) family of hash fuctios? I the aalysis of the previous sectio, the oly fact we used about radom hash fuctios was that Pr[h(x i ) = h(x j )] = 1/. Is it possible to costruct a small, practical subset of hash fuctios with this property? Thikig alog these lies, i 1978, Carter ad Wegma itroduced the otio of uiversal hashig: Cosider a family F of hash fuctios from U to {1, 2,..., }. We say that F is uiversal if, for every x i x j, for a h chose radomly from F, Pr[h(x i ) = h(x j )] 1/. Clearly the aalysis of the previous sectio shows that for ay uiversal family, the costat expected ruig time guaratee applies. The family of all hash fuctios is uiversal. Is there a simpler uiversal family? 1.4 A uiversal family of hash fuctios Suppose that the elemets of the U are ecoded as o-egative itegers i the rage {0,..., U 1}. Pick a prime p U. For a, b {0,... p 1}, cosider the family of hash fuctios where a [1, p 1] ad b [0, p 1]. 2 h a,b (x) = ax + b mod p mod Propositio 1. This family of hash fuctios F is uiversal. I order to prove this statemet, first, let s cout the umber of hash fuctios i this family F. We have p 1 choices for a, ad p choices for b, so F = p(p 1). I order to prove that F is uiversal, we eed to show that for a h chose radomly from F, Pr[h(x i ) = h(x j )] 1/. Sice there are p(p 1) hash fuctios i F, this is equivalet to showig that the umber of hash fuctios i F that map x i ad x j to the same output is less tha or equal to p(p 1). To show that this is true, first cosider how h a,b behaves without the mod. Call these fuctios f a,b : f a,b (x) = ax + b mod p The f a,b have the followig useful property: 3

4 Claim 1. For a give x 1, x 2, y 1, y 2 [0, p 1] such that x 1 x 2 there exists oly oe fuctio f a,b such that f a,b (x 1 ) = y 1 AND f a,b (x 2 ) = y 2 Proof. Solve the above two equatios for a ad b: ax 1 + b y 1 (mod p) ax 2 + b y 2 (mod p) By subtractig the two equatios, we get: a(x 1 x 2 ) y 1 y 2 (mod p) Sice p is prime ad x 1 x 2, the above equatio has oly oe solutio for a [0, p 1]. The b y 1 ax 1 (mod p) So we have foud the uique a ad b such that f a,b (x 1 ) = y 1 ad f a,b (x 2 ) = y 2. I the above proof, ote that a = 0 oly whe y 1 = y 2 = b. This is why we restrict a 0, we do t wat the hash fuctio mappig all elemets to the same value b. Now, we have show that for a give x 1, x 2, for each selectio of y 1, y 2 with y 1 y 2, there is exactly oe fuctio f a,b that maps x 1 to y 1 ad x 2 to y 2. So, i order to fid out how may fuctios h a,b map x 1 ad x 2 to the same value mod, we just eed to cout the umber of pairs (y 1, y 2 ) where y 1 y 2 ad y 1 y 2 (mod ). There are p possible selectios of y 1 for this pair, ad the (p 1)/ of the possibilities for y 2 will be equal to y 1 mod. (Covice yourself that this is true.) This gives a total of p(p 1) fuctios h a,b that map x 1 ad x 2 to the same elemet. So the p(p 1)/ P r[h a,b (x 1 ) = h a,b (x 2 )] F p(p 1) = p(p 1)() which meas the family F of the h a,b is uiversal, as desired. Wrappig up the discussio o hashig, if we pick a radom hash fuctio from this family, the the expected cost of ay hashig operatio is costat. Note that pickig a radom hash fuctio from the family simply ivolves pickig a, b sigificatly simpler tha pickig a completely radom hash fuctio. 2 Balls ad Bis A useful abstractio i thikig about hashig with radom hash fuctios is the followig experimet: Throw m balls radomly ito bis. (The coectio to hashig should be clear: the balls represet the keys ad the bis represet the hash buckets.) The balls ito bis experimet arises i several other problems as well, e.g., aalysis of load balacig. I the cotext of hashig, the followig questios arise about the balls ad bis experimet: How large does m have to be so that with probability greater tha 1/2, we have (at least) two balls i the same bi? This tells us how large our hash table eeds to be to avoid ay collisios. We will explore this at the ed of these otes. Suppose m = ; what is the maximum umber of balls that fall ito a bi? This tells us the size of the largest bucket i the hash table whe the umber of keys is equal to the umber of buckets i the table. We might explore this i the ext homework. = 1 4

5 No collisios The first questio is related to the so called birthday paradox: Suppose you have 23 people i a room. The (somewhat surprisigly) the probability that there exists some pair with the same birthday is greater tha 1/2! (This assumes that birthdays are idepedet ad radomly distributed.) 23 seems like a awfully small umber to get a pair with the same birthday. There are 365 days i a year! How do we explai this? Cosider throwig m balls ito bis. The expected umber of pairs that fall ito the same bucket is m(m 1)/2. (This follows from liearity of expectatio. Note that the probability that a fixed pair falls ito the same bucket is 1/.) Thus the probability that there is a collisio is upper bouded by the expected umber of collisios which is m(m 1)/2. (Covice yourself that this is true.) O the other had, we ca also show that the probability that all m balls fall ito distict bis is at most e m(m 1)/2 : Proof. Now, we use the fact that (1 x) e x : P [o collisios] = (1 i ) m 1 i=1 (1 i ) e i/ So P [o collisios] m 1 i=1 e i/ P [o collisios] e m 1 i=1 i/ P [o collisios] e ( m(m 1)/(2)) For m about (2 l 2) 1.18 this probability is less tha 1/2, i.e. the probability of a collisio is greater tha 1/2. This is a useful desig priciple to keep i mid: If we wat to desig a hash table with o collisios, the the size of the hash table should be larger tha the square of the umber of elemets we eed to store i it. For our purposes i this ote, isistig o o collisios meas that the umber of elemets i the hash table ca oly be a small fractio of the hash table size which is quite wasteful. The birthday problem calculatio is useful i other cotexts. Here is a applicatio: Suppose we assig radom b-bit IDs to m users. How large does b have to be to esure that all users have distict IDs with probability 1 δ. Here δ > 0 is a give error tolerace. Assigig b-bit IDs is idetical to mappig to = 2 b buckets. The birthday problem calculatio shows us that the probability of a collisio is at most m 2 /2 = m 2 /2 b+1. We should set b large eough such that this boud is at most δ. Thus b should be at least 2 log m 1 + log(1/δ). 5